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Experimental  Assessment  Parameters  and  Procedures 
for  Characterisation  of  Advanced  Thermal  Imagers 

(RTO  TR-075(II)  /  SET-015) 


Executive  Summary 


The  infrared  community  has  entered  an  era  where  staring  imagers  are  becoming  the  norm  and  not  the 
exception.  Staring  array  sensors  are  quickly  making  their  way  into  target  acquisition  and 
surveillance/reconnaissance  applications.  The  large  detector  characteristics  and  improved  optical 
quality  of  these  systems  make  for  undersampled  image  transfer  with  high  sensitivity.  The  classical 
MTF  and  MRTD  resolution  and  visual  performance  parameters,  respectively,  do  not  exist  for 
undersampled  imagers.  In  addition,  the  higher  sensitivity  of  these  systems  requires  that  new  noise 
measures  be  implemented. 

The  objective  of  this  study  undertaken  by  the  RTO  SET  Panel  Task  Group  12  (TG12)  was  the 
development  and  investigation  of  experimental  assessment  parameters  and  measurement  techniques 
required  for  characterising  advanced  staring  or  micro-scanned  thermal  imagers.  This  study  is 
complementary  to  a  previous  TG12  report  entitled  “Modelling  of  Undersampled  Infrared  Imaging 
Systems”  which  is  published  as  a  separate  RTO  Publication  (RTO-TR-075(I)). 

TG12  collectively  and  TG12  member  countries  have  developed  and  investigated  a  number  of  measures 
and  measurement  techniques  to  characterise  advanced  thermal  imagers.  They  are  presented  in  this 
report.  These  measures  and  techniques  allow  assessment  of  basic,  system-relevant  sensor  parameters, 
such  as  spatial  resolution  and  noise,  as  well  as  overall  sensor  performance. 

Numerous  measures  and  techniques  are  now  available  to  characterise  spatial  information  transfer  and 
noise  in  advanced  thermal  imagers.  Also,  various  figures  of  merit  and  measurement  techniques  were 
developed  to  assess  the  overall  device  performance.  Some  of  the  new  figures  of  merit,  such  as 
Minimum  Temperature  Difference  Perceived  (MTDP),  are  evolutions  of  the  classical  MRTD  figure  of 
merit.  Other  techniques  for  overall  performance  assessment  use  triangles  and  double-slits  as  test 
patterns.  These  techniques  were  developed  to  reduce  possible  measurement  inaccuracies  introduced  by 
the  subjective  nature  of  assessment  techniques  which  use  standard  4-bar  test  patterns,  and  have  benefits 
which  should  further  be  investigated.  All  figures  of  merit  will  remain  in  use  for  further  evaluation, 
which  is  strongly  recommended. 

Techniques  to  assess  overall  device  performance  without  human  observer  respond  to  the  continued 
interest  of  the  IR  community  to  have  fast  and  reliable  assessment  techniques.  Such  techniques  are  still 
under  development. 

Summary  of  Recommendations: 

1.  Establish  a  board  of  experts  and  an  AGGRESSIVE  program  for  developing/evaluating  techniques 
for  modelling,  measurements,  and  range  performance  predictions  of  undersampled  imagers  with  the 
goal  of  drafting  STANAG(s)  for  imager  performance.  Realize  that  this  is  a  significant  effort  that 
requires  nations  to  participate  in  laboratory  evaluations  of  techniques,  so  nations  must  be  willing  to 
invest  time  and  effort  in  a  development/evaluation  program.  It  should  not  be  the  part-time  goal  of  a 
Technical  Group. 

2.  Note  that  for  the  immediate  future,  there  will  be  mis-applications  of  the  current  STANAG  methods 
to  undersampled  imagers.  Staring  and  scanning  infrared  sensors  are  unfairly  compared  using  the 
current  STANAG  methods.  It  is  recommended  that  the  MTDP  method  or  the  NVTherm/DMRT 
(dynamic  MRTD)  method  be  used  until  further  evaluation  can  be  performed. 
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3.  Standardise  definitions  and  measurement  techniques  for  assessing  resolution  and  sensitivity  of 
advanced  imagers. 

4.  Review  and  update  STANAGs  4347,  4349,  and  4350  for  use  with  well-sampled  imaging  systems. 
There  have  been  numerous  improvements  in  MRTD  modelling,  measurements,  and  range 
performance  over  the  past  10  years. 

5.  Develop  “objective”  techniques  to  assess  overall  device  performance  without  human  observer.  This 
is  in  the  continued  interest  of  the  IR  community  and  hence  should  be  further  encouraged. 


Parametres  et  procedures  devaluation 
experimentale  pour  la  caracterisation 
d’imageurs  thermiques  avances 

(RTO  TR-075(II)  /  SET-015) 


Synthese 

Pour  les  specialistes  de  1’ infrarouge,  l’epoque  qui  commence  est  celle  ou  les  imageurs  a  mosaique 
infrarouge  sont  maintenant  la  norme  et  non  plus  l’exception.  Aujourd’hui,  les  capteurs  fixes  trouvent  de 
plus  en  plus  d’ applications  dans  les  domaines  de  l’acquisition  d’objectifs  et  la  surveillance/la 
reconnaissance.  Les  grandes  caracteristiques  de  detection  et  la  qualite  optique  amelioree  de  ces  systemes 
permettent  de  realiser  des  transferts  d’ images  sous-echantillonnees  avec  une  grande  sensibilite.  Les 
parametres  de  resolution  et  de  performance  visuelle  des  MTF  et  MTRD  classiques  n’existent  pas  pour  les 
imageurs  sous-echantillonnes.  En  outre,  la  plus  grande  sensibilite  de  ces  systemes  necessite  la  realisation  de 
nouvelles  mesures  de  bruit. 

Cette  etude,  effectuee  par  le  groupe  de  travail  No.  12  de  la  commission  SET  de  la  RTO,  a  eu  pour  objectif 
d’examiner  et  de  developper  des  parametres  experimentaux  devaluation,  ainsi  que  des  techniques  de 
mesure  pour  la  caracterisation  d’imageurs  thermiques  a  mosaique  infrarouge  et  a  micro-balayage  avances. 
Cette  etude  vient  completer  le  precedent  rapport  de  TGI 2  intitule  «  La  modelisation  de  systemes  d’imagerie 
infrarouge  sous-echantillonnes  »  qui  est  edite  sous  forme  de  la  publication  RTO-TR-075(I). 

TG12,  de  faqon  collective,  et  les  pays  membres  de  TG12  de  faqon  individuelle,  ont  examine  et  developpe 
un  certain  nombre  de  mesures  et  techniques  de  mesure  qui  permettent  de  caracteriser  les  imageurs 
thermiques  avances.  Elies  sont  presentees  dans  ce  rapport.  Ces  mesures  et  techniques  permettent  d’evaluer 
les  parametres  de  base  des  systemes  de  detection,  tels  que  la  resolution  spatiale  et  le  bruit,  ainsi  que  les 
performances  globales  des  capteurs. 

Un  grand  nombre  de  mesures  et  de  techniques  sont  desormais  disponibles  pour  la  caracterisation  du 
transfert  spatial  des  informations  et  du  bruit  dans  les  imageurs  thermiques  avances.  Aussi,  differents 
facteurs  de  qualite  et  techniques  de  mesure  ont  ete  developpes  afin  d’evaluer  les  performances  globales  des 
dispositifs.  Certains  nouveaux  criteres  de  merite  tels  que  la  difference  minimale  de  temperature  perque 
(MTDP),  sont  des  evolutions  du  facteur  de  merite  MRTD  classique.  D’autres  techniques  devaluation  des 
performances  globales  font  appel  a  des  mires  composees  de  triangles  et  de  trous  de  Young.  Ces  techniques 
ont  ete  elaborees  dans  le  but  de  reduire  d’eventuelles  erreurs  de  precision  introduites  par  la  nature 
subjective  des  techniques  devaluation  basees  sur  des  mires  classiques  a  quatre  barres,  et  meritent  un 
examen  plus  approfondi.  Tous  les  facteurs  de  merite  seront  retenus  aux  fins  devaluations  ulterieures,  qui 
sont,  d’ailleurs,  fortement  recommandees. 

Des  techniques  destinees  a  evaluer  les  performances  globales  des  systemes,  sans  la  presence  d’observateurs 
humains  repondraient  a  la  demande  exprimee  par  la  communaute  IR,  de  techniques  devaluation  rapides  et 
sures.  Elies  sont  encore  en  cours  de  developpement. 

Resume  des  recommandations 

1.  Un  comite  de  specialistes,  avec  un  programme  dynamique  de  developpement/d’evaluation  de  techniques 
de  modelisation,  de  mesure,  et  de  prevision  de  performances  en  portee  d’imageurs  sous-echantillonnes, 
en  vue  de  la  redaction  de  STANAG’s  sur  les  performances  des  imageurs  devrait  etre  etabli.  II  faut  savoir 
que  ceci  demandera  des  efforts  significatifs.  En  particulier,  il  faudra  demander  aux  pays  de  participer  a 
des  evaluations  en  laboratoire  et  ils  devront  done  etre  prets  a  s’investir  pour  executer  un  programme  de 
developpement/d’evaluation.  II  ne  devrait  pas  s’agir  d’un  projet  a  temps  partiel  pour  un  groupe 
technique. 


2.  II  faut  noter  qu'a  court  terme,  il  faut  s’attendre  a  des  applications  erronees  des  methodes  preconisees  par 
les  STANAGs  actuels.  Les  comparaisons  qui  sont  actuellement  faites  entre  les  capteurs  a  mosaique 
infrarouge  et  les  capteurs  a  balayage  infrarouge,  selon  les  methodes  preconisees  par  les  STANAGs,  ne 
sont  pas  justes.  En  attendant  d’autres  evaluations,  il  serait  preferable  d’appliquer  soit  la  methode  MTDP, 
soit  la  methode  NVTherm/DMRT  (MRTD  dynamique). 

3.  Les  definitions  et  les  techniques  de  mesure  pour  revaluation  de  la  resolution  et  la  sensibilite  des 
imageurs  avances  devraient  etre  normalisees. 

4.  Les  STANAGs  4347,  4349  et  4350  pour  application  aux  systemes  d’imagerie  correctement 
echantillonnes  doivent  etre  revus  et  mis  a  jour.  De  nombreuses  ameliorations  ont  ete  apportees  a  la 
modelisation,  aux  mesures  et  aux  performances  en  distance  du  MRTD  au  cours  des  dix  dernieres  annees. 

5.  Il  s’agit  maintenant  d’elaborer  des  techniques  «  objectives  »,  permettant  d’evaluer  les  performances 
globales  des  systemes  sans  intervention  humaine.  Une  telle  demarche  conforterait  les  interets  a  long 
terme  des  specialistes  de  1’IR  et  devrait  etre  encouragee. 
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Chapter  1 
Introduction 


The  battle-winning  capability  provided  by  infrared  technology  has  been  published  in  recent  years  by 
worldwide  media  coverage  of  major  military  conflict.  The  advantages  provided  by  these  technologies  are 
now  well  established  and  understood  by  military  strategists. 

To  respond  to  the  demand  for  product  superiority  in  infrared  technology,  a  new  generation  of 
advanced  infrared  focal  plane  array  detectors  (FPA)  have  been  developed  in  NATO,  most  notably  in 
France,  Germany,  the  UK  and  US.  Many  different  types  of  FPAs  are  now  available  from  uncooled  arrays 
(e.g.  bolometer)  to  cooled  arrays  using  a  wide  range  of  photosensitive  materials  and  technologies  (e.g. 
CMT,  InSb,  PtSi,  QWIP). 

These  advanced  state  of  the  art  IR  technologies  are  now  becoming  available  in  a  number  of  IR 
systems  either  entering  service  now,  or  planned  to  enter  service  in  the  near  future.  They  are  offering 
increased  performance  with  the  added  benefits  of  improved  reliability  and  lower  life  cycle  costs. 

In  response  to  the  perceived  need  to  coordinate  a  common  approach  to  the  assessment  and 
characterisation  of  these  advanced  systems,  TGI 2  was  tasked  with  developing  an  approach  to  the  analysis 
of  advanced  IR  FPA  systems.  This  report  is  the  advances  produced  by  TGI 2  in  assessing  these  IR 
systems.  Performance  measures  developed  for  1st  generation  thermal  imagers  that  can  also  be  used  without 
modifications  for  advanced  systems  were  not  addressed  and  are  not  reviewed  in  this  paper. 

The  puipose  of  this  report  is  to  inform  the  NATO  community  of  the  conclusions  of  NATO  RTO- 
SET/TG12’s  work  concerning  the  development  of  a  common  approach  to  the  experimental 
characterisation  and  assessment  of  advanced  IR  systems,  to  indicate  areas  where  further  convergence  of 
ideas  and  techniques  is  needed,  and  to  recommend  the  preparation  of  future  NATO  STANAGs  covering 
the  definition,  assessment  and  characterisation  of  advanced  IR  imagers  in  those  areas  where  formal 
agreement  now  exists  within  TG12. 

In  a  companion  report1  we  inform  the  NATO  community  on  the  status  of  the  range  performance 
modelling  of  advanced  thermal  imagers,  and  the  activities  TGI 2  has  undertaken  to  validate  current  target 
acquisition  models. 


1.1  DEFINITION  OF  ADVANCED  SYSTEMS 

It  is  important  at  the  outset  to  define  clearly  what  TGI 2  means  by  an  advanced  IR  system  and  what  the 
group  has  excluded  from  its  current  study.  In  the  context  of  the  assessment  and  characterisation  studies 
conducted  within  the  group,  an  advanced  system  has  been  defined  as  one  that  utilises  infrared  focal  plane 
arrays  where  the  detector  signals  are  integrated  on  the  chip  and  multiplexed  before  read-out.  Inherent 
feature  of  advanced  systems  is  that  the  information  is  sampled  in  both  directions. 

Examples  of  advanced  infrared  focal  plane  arrays  include  two-dimensional  arrays  of  e.g.  640  x  512 
detectors  and  linear  detector  arrays  with  e.g.  576  x  7  detectors.  Two-dimensional  arrays  stare  at  the 
infrared  scene  whereas  linear  detector  arrays  are  scanned  mechanically  by  the  use  of  mirrors  etc  across  the 
FOV.  Further  distinction  may  be  made  between  two-dimensional  detector  arrays  that  are  purely  staling 
and  those  that  are  micro-scanned  (or  dithered). 

The  generic  type  of  'advanced'  infrared  imager  considered  by  TG12  is  shown  in  Fig.  1.1.  It  is  a 
staling  imager  with  a  two-dimensional  FPA.  This  type  of  imager  is  undersampled  and  has  non-neglectable 
fixed  pattern  noise.  These  features,  which  are  described  in  Section  2,  constitute  the  new  challenges  in 
assessing  advanced  systems. 


2 


□□□□□□□□□□□a 
□□□□□□□□□□□a 
□□□□□□□□□□□a  n 
□□□□□□□□□□□a 
□□□□□□□□□□□a 

m 

Fig  1.1:  Generic  type  of  an  advanced  system  as  studied  by  TGI 2. 

Advanced  system  includes  cooled  and  uncooled  technologies  and  the  many  different  silicon 
multiplexers  and  charge  storage  and  read-out  techniques,  e.g.  CCD,  CMOS  etc.  The  techniques  and 
methodologies  developed  and  tested  by  the  group  are  designed  to  be  generic  and  are  therefore  not 
sensitive,  for  example,  to  the  type  of  silicon  multiplexer  used,  or  the  infrared  diode  structure  as  this  would 
restrict  the  usefulness  of  the  tests  and  techniques  to  a  narrow  range  of  technologies. 

The  group  has  not  extended  the  study  to  an  analysis  of  focal  plane  arrays  with  retinal  or  adaptive 
processing  on  the  focal  plane  neither  are  functions  such  as  on  focal  plane  ADC  considered.  These  areas 
were  beyond  the  remit  of  the  group. 

1.2  STRUCTURE  OF  THE  REPORT 

The  next  sections  are  devoted  to  the  exposition  of  measures,  figures  of  merit  and  measurement  techniques 
developed  or  studied  by  TG12  and  its  predecessor  group,  RSG16  of  AC/243,  and  used  to  characterise 
advanced  systems.  They  are  divided  in  three  main  categories,  which  are  treated  in  separate  sections. 

Section  2  describes  measures  characterising  the  spatial  resolution  of  advanced  imagers.  First  it  is 
outlined  why  the  system  MTF  cannot  be  used  to  characterize  the  spatial  resolution  of  undersampled 
imagers.  The  newly  developed  assessment  parameters  and  measurement  techniques  that  take  into  account 
the  special  requirements  imposed  by  the  sampling  process  in  advanced  systems  are  considered  here. 

Section  3  describes  measures  characterising  the  thermal  sensitivity  of  the  imager.  Attention  is  given 
to  characterize  the  temporal  and  spatial  noise  in  advanced  FPA  systems  including  bad  pixel 
characterization. 

Section  4  deals  with  figures  of  merit  characterising  overall  device  performance:  The  sampling  of  the 
detector  signals  as  well  as  spatial  noise  due  to  non-uniformities  affect  the  performance  of  advanced 
imagers.  Overall  figures  of  merit  that  cope  with  these  features  are  presented  and  compared  to  the  classical 
MRTD  figure  of  merit.  Subjective  measures  as  well  as  objective  techniques  are  addressed. 

The  measured  are  discussed  in  Section  5.  General  conclusions  and  suggestions  for  further  research 
and  recommendations  are  given  in  Section  6.  For  more  technical  information,  original  papers  are  reported 
in  the  REFERENCES. 
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Chapter  2 

Assessment  of  Spatial  Information  Transfer 

2.1  THE  LIMITS  OF  THE  MTF  CONCEPT 


2.1.1.  The  MTF  concept 

The  ability  to  reconstruct  spatial  information  of  a  scene  is  related  to  the  spatial  resolution  of  the  imager. 
Tools  to  describe  the  spatial  resolution  of  thermal  imagers  are  derived  from  linear  filter  theory. 

Assuming  that  the  imager  is  a  linear,  space-  and  time-invariant  system,  the  spatial  resolution  of  the 
imager  is  described  by  its  Point  Spread  Function,  PSF,  and  its  Optical  Transfer  Function,  OTF,  which  is 
the  Fourier  transform  of  the  PSF.  Modulus  and  phase  of  the  OTF  are  defined  as  Modulation  Transfer 
Function,  MTF,  and  phase  transfer  function,  PTF,  of  the  system.  PSF  and  OTF/MTF  are  two-dimensional 
functions. 

The  Line  Spread  Function,  LSF,  is  a  one-dimensional  function.  It  is  given  by  integrating  the  PSF  in 
one  direction: 

LSF(  x  )  =  \PSF(x,y)dy  . 


The  Fourier  transform  of  the  LSF  is  the  one-dimensional  OTF,  its  modulus  the  one-dimensional 
MTF.  Most  measurement  techniques  derive  the  one-dimensional  MTF  from  an  LSF  measurement  using  a 
slit  source  as  test  pattern.  The  PTF  is  not  used  to  assess  thermal  imager  performance. 

The  MTF  is  a  powerful  parameter  to  assess  1  i near  and  space-invariant  imaging  systems  since  the 
MTF  of  a  complex  system  is  the  product  of  the  MTFs  of  its  incoherently  coupled  components.  Theory  and 
measurement  of  the  MTF  of  imaging  systems  is  treated  in  a  STANAG. 2  3 


2.1.2.  Applicability  of  the  MTF  concept  to  advanced  thermal  imagers 

In  order  to  apply  the  MTF  concept  to  a  thermal  imager  it  must  behave  like  a  linear  and  isoplanatic  system. 
Within  a  limited  range  of  scene  temperatures  the  imager  always  has  a  linear  response  to  the  incoming 
signal.  This  linear  range  depends  on  the  gain  and  offset  setting  of  the  imager  and  can  be  derived  from  the 
measurement  of  the  Signal  Transfer  Function  (SiTF).  4 

The  imager  is  isoplanatic  if  the  point  spread  function  is  space-invariant;  i.e.  it  does  not  change 
within  the  field  of  view  (FOV),  or  at  least  paid  of  it.  While  first  generation  (scanning)  imagers  with  analog 
signal  processing  can  be  considered  space-invariant  in  scan  direction,  advanced  thermal  imagers, 
especially  staring  imagers,  usually  are  not  space-invariant  in  any  direction.  This  is  due  to  the  insufficient 
sampling  of  spatial  information  in  both  directions.  Individual  components  of  the  imager,  however,  and 
groups  of  components  can  be  considered  space-invariant. 

With  respect  to  the  MTF  characterization,  imager  components,  which  act  on  the  signal  before 
sampling,  can  be  considered  as  prefilter  components.  Imager  components  that  act  on  the  signal  after 
sampling  can  be  considered  as  postfilter  components.  Normally,  these  components  are  space-invariant  and 
hence  can  be  described  by  MTFs: 


Prefilter  MTF:  The  product  of  the  MTFs  of  all  incoherently  coupled  prefilter  components.  (E.g. 

front  optics  and  detector  elements). 
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Postfilter  MTF:  The  product  of  the  MTFs  of  all  incoherently  coupled  postfilter  components  (e.g. 
video  and  display). 

The  MTF  concept,  however,  is  not  suited  to  assess  the  spatial  transfer  characteristics  of  an  advanced 
imager  as  a  whole.  This  is  described  in  more  detail  in  the  next  paragraphs.  Alternative  descriptors  are 
needed. 


2.1.3.  The  sampling  of  information  in  advanced  imagers 

A  two-dimensional  focal  plane  detector  array  is  a  device  that  consists  of  detector  elements  with  a 
particular  size  and  shape,  separated  by  a  given  distance,  see  Fig.  2.1.  The  distance  between  the  centres  of 
two  adjacent  detectors  is  the  detector  pitch. 
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Fig  2. 1 :  View  of  the  physical  structure  of  a  two-dimensional  FPA 


The  sampling  distances  are  the  distances  between  subsequent  horizontal  and  samples  taken  by  the 
imager.  In  staling  imagers  the  sampling  distance  in  either  direction  equals  the  detector  pitches.  In  micro- 
scanned  imagers,  the  sampling  distance  in  either  direction  is  given  by  the  pitch  divided  by  the  number  of 
micro-scan  steps  in  that  direction.  The  sampling  frequency  is  the  inverse  of  the  sampling  distance. 


2.1.4.  Sampling  and  undersampling 

The  sampling  theorem  deals  with  the  capacity  of  a  sampled  system  to  transmit  information.  In  case  of 
optical  information  this  capacity  means  the  ability  of  the  system  to  reproduce  the  spatial  frequencies  that 
form  a  particular  scene.  The  principal  result  of  this  theorem  is  to  establish  an  upper  limit  for  frequencies 
that  can  be  faithfully  transmitted  and  reconstructed  from  the  original  signal.  This  limit  is  often  called  the 
Nyquist frequency  which  is  given  by, 

fNyquist  = 1/ 2b . 
where  b  is  the  sampling  distance. 

Mathematically,  in  sampling  systems,  the  spectrum  of  the  original  signal  is  replicated  at  intervals 
fs ,  where  f\  is  the  sampling  frequency,  see  Fig.  2.2.  A  signal  at  frequency  /  re-appears  as  a  pseudo¬ 
signal  at  the  frequencies  nfs  -  f  and  nfs  +  /  ,  with  m  any  integer  number. 

If  the  spectrum  of  the  original  signal  contains  frequencies  higher  than  the  Nyquist  frequency,  the 
original  spectrum  and  the  replicated  spectra  overlap.  In  such  a  case,  the  original  signal  can  by  no  means  be 
reconstructed  faithfully.  An  imager  who  allows  that  the  original  spectrum  overlaps  with  the  back-folded 
spectrum  is  called  undersampled.  The  imager  is  undersampled  if  the  prefilter  MTF  is  not  equal  to  zero  at 
Nyquist  frequency. 
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Fig  2.2:  The  spectrum  of  a  sampled  signal:  The  original  spectrum  is  replicated  at  multiples  of  the 
sampling  frequency.  If  the  prefilter  MTF  of  the  imager  is  band-limited  at  a  frequency 
lower  than  the  Nyquist  frequency  (top  figure)  the  image  is  properly  sampled.  If  this  is 
not  the  case  (bottom  figure)  the  imager  is  undersampled. 


The  original  signal  can  be  reconstructed  faithfully  only  if  (1)  the  signal  is  band  limited  at  a 
frequency  lower  than  the  Nyquist  frequency  and  if  (2)  a  postfilter  is  used  which  eliminates  all  back- folded 
frequency  components  higher  than  the  Nyquist  frequency.  If  these  conditions  are  not  met,  artefacts  appear 
in  the  reconstructed  image.  This  is  called  aliasing. 

The  imager  is  isoplanatic,  or  shift  invariant,  if  the  “shape”  of  the  PSF  does  not  depend  on  the 
particular  position  on  the  object  plane.  If  the  object  to  be  viewed  is  moved,  the  image  remains  the  same. 
This  is  not  true  for  an  undersampled  imager.  The  PSF  depends  on  the  position  in  object  space;  the  system 
is  not  shift  invariant.  More  specifically,  the  PSF  depends  on  the  position  of  the  point  source  relative  to  the 
detector  array.  This  relative  position  is  called  the  phase,  see  Fig.  2.3. 


Maximum  signal  Minimum  signal 

Fig  2.3:  Different  phases  between  target  and  detector  array  produce  different  signals. 


Due  to  the  periodicity  of  the  detectors  in  the  array,  the  changes  in  PSF  due  to  phasing  effects  are 
periodic,  too.  A  point  source  is  viewed  under  the  best  conditions  when  the  centre  of  the  point  is  in  phase 
with  the  centre  of  a  detector.  The  worst  case  is  given  when  the  point  source  is  located  between  the 
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detector  elements.  Generally,  contrast  and  modulation  of  a  signal  depend  on  its  phase  with  respect  to  the 
detectors. 


2.1.5.  New  performance  measures  and  techniques 

Although  in  case  of  undersampling,  the  overall  system  MTF  is  not  defined  and  hence  cannot  be  measured, 
the  MTF  concept  still  can  be  used  to  characterize  paid  of  the  imager. 

The  prefilter  MTF  plays  an  important  role  in  the  TRM35  and  NVTherm6  models.  If  measured,  it 
allows  to  calculate  the  actual  amount  of  in-band  aliasing  of  the  imager  (NVTherm)  and  to  simulate  the 
actual  sampling  of  information  (TRM3).  The  postfilter  MTF  allows  assessing  how  the  digitised  signals  are 
actually  reconstructed.  If  both  pre-  and  postfilter  MTFs  are  measured  the  amount  of  out-of-band  aliasing 
in  the  displayed  image  can  be  calculated  (NVTherm)  and  the  image  of  an  input  signal  can  be  reconstructed 
(TRM3). 

Special  techniques  are  required  and  were  developed  to  perform  MTF  measurements  in 
undersampled  imagers.  These  techniques  are  described  in  the  following  paragraphs.  For  the  overall 
assessment  new  concepts  were  developed  (AMOP  and  double-slit  technique)  that  are  described  in  the 
remaining  paragraphs  of  this  section. 


2.2  BEST  AND  WORST ‘MTF’ 

In  case  of  isoplanatic  systems  the  LSF  can  be  measured  by  either  moving  the  test  source  and  analysing  the 
signal  at  a  fixed  position  or  vice  versa.  The  result  is  the  same  and  leads  to  the  (one-dimensional)  MTF. 

The  standard  technique  to  measure  the  horizontal  LSF  in  1st  generation  thermal  imagers  with  analog 
signal  output  is  to  keep  the  line  source  fixed  and  to  record  and  to  analyse  the  output  signal.  The  modulus 
of  the  normalized  Fourier  transform  of  this  signal  gives  the  system. 

If  this  procedure  is  applied  to  advanced  undersampled  thermal  imagers  the  measured  signal  depends 
on  the  phase  between  test  slit  and  detector  array,  as  described  in  the  previous  paragraph.  The  technique  is 
still  used  by  some  laboratories  to  characterise  the  performance  of  undersampled  thermal  imagers. 
Normally,  measurements  are  made  at  best  and  worst  phase  (see  above).  The  modulus  of  the  Fourier 
transform  of  the  measured  and  normalised  line  signal  is  designated  best  MTF’  and  worst  MTF.  ’ 

Although  these  measurements  are  no  MTF  measurements  useful  information  can  be  extracted:  The 
shape  of  both  best  and  worst  MTF  is  governed  by  the  postfilter,  most  especially  by  the  reconstruction 
function.  The  type  of  reconstruction  function  used  can  be  easily  estimated  from  this  measurement.  Also, 
comparison  of  best  and  worst  MTF  allows  qualitative  assessment  of  the  undersampling  of  the  imager.  In 
addition,  from  the  signal  difference  between  the  peak  signals  at  best  and  worst  phase  gives  a  measure  for 
the  effective  filling  factor  of  the  imager. 


2.3  SCANNING  SLIT  METHOD 

In  the  Scanning  slit  method7  the  slit  source  is  moved  and  the  reproduced  line  signal  is  analysed  at  a  fixed 
position  in  the  output  signal,  whether  digital  or  video  output,  see  Fig.  2.4.  The  signal  at  this  position  is 
recorded  as  function  of  the  slit  displacement,  which  gives  the  line,  spread  function. 
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Signal  is  analyzed 
at  a  fixed  position 


Fig.  2.4:  Scanning  slit  method.  The  test  slit  is  moved  and  the  signal  is  analyzed  at  a  fixed  position 
in  the  output  signal.  The  signal  as  function  of  the  slit  displacement  gives  the  line  spread 
function. 

The  method  can  be  applied  to  all  types  of  sampled  imagers,  scanning,  staring  or  micro-scanned.  The 
advantage  of  this  method  is  that,  with  few  exceptions,  the  measurement  result  is  unique  and  can  be 
interpreted:  If  the  measurement  is  made  in  the  digital  domain,  the  measurement  leads  to  an  MTF,  which  is 
given  by 

MTF  =  MTF  ■  MTF 

1V11  1  scanning  slit  1V1 1  1  pref  lrl  1  1  dig 

where  MTFpref  is  the  MTF  of  the  prefilter  and  MTFdig  the  MTF  of  any  digital  filter  used.  Undersampled 
imagers  usually  have  no  digital  filter  and,  therefore,  the  measurement  normally  gives  the  prefilter  MTF. 

If  measured  in  the  video  signal  the  technique  leads  to  a  result,  which  is  given  by: 

MTF 'scanning  slit  =  MTF pref  '  MTF dig  '  C<j)  , 

where  c  is  a  function  of  spatial  frequency  r.  c  depends  on  the  postfilter  MTF  and  is  equal  to  one  if  the 
postfilter  MTF  does  not  introduce  additional  signal  degradation. 

One  advantage  of  the  scanning  slit  technique  is  that  using  just  one  single  detector  as  probe  residual 
detector  non-uniformities  do  not  affect  the  measurement.  The  displacement  of  the  slit  source  can  be  made 
with  very  small  increments,  thus  improving  the  accuracy  of  the  measured  line  spread  function. 

In  order  to  obtain  a  sufficiently  high  signal-noise-ratio  the  signal  at  each  slit  position  must  be 
measured  repeatedly  in  subsequent  frames  and  averaged.  If  the  measurement  is  made  in  the  analog  video 
signal  no  expensive  digital  data  acquisition  equipment  is  needed. 


2.4  DMTF  METHOD  (DISCRETE  MODULATION  TRANSFER  FUNCTION) 

The  DMTF  method8  is  preferably  applied  to  staring  imagers  with  digital  output.  The  basic  approach  is  to 
record  the  test  slit  image  at  different  equidistant  phases  and  to  interweave  the  signal  recorded  at  each 
phase  position,  thus  increasing  the  effective  sampling  frequency. 


Assume  that  the  pitch  in  the  focal  plane  array  be  b.  The  sampling  frequency  is  then  1/b.  The  test  slit, 
or  the  imager,  is  moved  in  n  discrete  steps  at  step  intervals  Ax  given  by 


n 


At  each  step  the  test  slit  image  is  recorded  at  the  digital  output.  The  recorded  sampled  image  signals 
are  then  interwoven.  This  gives  a  new  sampled  test  image  signal  with  a  sampling  distance  of  Ax  or  a 


sampling  frequency  that  is  n  times  higher  than  the  original  sampling  frequency  of  the  imager.  The 
approach  is  illustrated  in  Fig.  2.5. 


output  samples 


a)  1st  recording 
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output  samples 

b)  2nd  recording  with  the  test  slit  displaced  by 
l/2b 


c)  Interwoven  signal 

Fig.  2.5:  Illustration  of  the  DMTF  method  with  n=2.  The  test  image  signal  is  recorded  at  two 
equidistant  phase  positions  (top  part).  The  two  recordings  are  then  interlaced  (bottom 
right).  This  gives  a  sampled  test  slit  image  with  a  sampling  frequency  that  is  two  times 
higher  than  the  original  one. 


The  number  of  steps  n  is  chosen  in  such  a  way  that  the  Nyquist  criterion  is  satisfied  for  the  highest 
frequency  transmitted  by  the  system  (usually  the  optics  cut-off). 

The  DMTF  method  is  a  fast  technique  to  assess  staring  or  micro-scanned  imagers,  preferably  at  the 
digital  video  output.  Contrary  to  the  scanning  slit  technique,  several  detectors  contribute  to  the  measured 
line  signal.  It  is  assumed  that  these  detectors  have  the  same  performance.  Residual  detector  non¬ 
uniformity  must  be  removed  in  order  to  obtain  an  un-biased  signal. 


2.5  THE  TILTED  SLIT  METHOD 

A  slit  (or  edge)  is  positioned  at  a  small  angle  relative  to  the  detector  columns  or  lines,  see  Fig.  2.6. 
Assume  that  the  slit  is  tilted  by  an  angle  a  relative  to  a  horizontal  line  of  detectors.  Between  the  centres  of 
two  adjacent  detector  elements  the  slit  is  displaced  in  vertical  direction  by  Ay,  with 

Ay  =b-  tan(  Ot ) 

where  b  is  the  horizontal  detector  pitch.  The  output  signal  is  analysed  in  horizontal  direction. 

With  this  technique  one  obtains  samples  of  the  vertical  line  spread  function  at  sample  intervals  Ay.  If  the 
slit  is  tilted  relative  to  a  vertical  line  of  detectors  the  output  signal  is  analysed  in  vertical  direction  to 
obtain  samples  of  the  horizontal  line  spread  function.  The  smaller  the  tilt  angle  a,  the  smaller  is  the 
sample  interval.  Fig.  2.7  shows  the  image  of  a  tilted  slit  as  an  example. 
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Fig.  2.6:  The  tilted  slit  method.  The  test  slit  (or  edge)  is  tilted  relative  to  the  orientation  of  the 
detector  elements. 


Fig.  2.7:  Example  of  an  image  taken  with  a  tilted  slit. 

The  tilted  slit  (or  edge)  method  is  a  fast  technique.  The  measurement  can  be  made  at  the  digital 
video  output  or  in  the  analog  video  signal.  The  tilt  angle  a  must  be  known  with  high  accuracy.  Residual 
detector  non-uniformities  affect  the  signal  the  same  way  as  in  the  DMTF  method. 


2.6  GLSF  METHOD  (GENERALIZED  LINE  SPREAD  FUNCTION). 

The  GLSF  technique9  uses  a  periodic  test  pattern  of  thin  lines  which  are  separated  by  a  distance  d,  see  Fig. 
2.8  This  distance  is  chosen  such  that 


where  s  is  the  sampling  distance  and  n  an  integer  that  depends  on  the  cut-off  frequency  of  the  front  optics 
of  the  imager.  The  frequency  spectrum  of  this  test  pattern  has  discrete  values  at  frequencies  q  with 

i 


r  = 


(  n 

n  H — 

l  2 ) 


with  i  =  1,2, 3,... 


Due  to  the  factor  1/2  the  back-folded  frequency  spectrum  does  not  interfere  with  the  original 
spectrum.  One  obtains  a  measure  free  of  aliasing  from  zero  spatial  frequency  up  to  the  sampling 
frequency. 
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Fig.  2.8:  View  of  a  GLSF 
target 


Fig.  2.9:  Fourier  spectrum  of  a  GLSF  target  obtained  at  the  output  of  a 
thermal  imager.  The  spatial  frequency  is  normalized  with  the 
sampling  frequency.  The  arrows  designate  useful  frequencies 


The  GLSF  is  the  signal  obtained  perpendicular  to  the  line  pattern.  Due  to  the  periodicity  of  the  test 
pattern  and  the  choice  of  the  distance  between  the  lines,  the  Fourier  transform  of  the  GLSF  at  frequencies 
r,,  see  Fig.  2.9,  is  not  affected  by  the  sampling  of  the  imager  and  hence  practically  the  same  at  different 
positions  over  the  array. 

2.7  FRACTAL  TEST  PATTERN  TECHNIQUE 

Two  MTF  assessment  methods10  have  been  designed  which  use  large  area  grey-level  test  targets  that  are 
composed  of  a  set  of  wavelet  patterns  distributed  according  to  a  fractal  algorithm. 

The  first  method  uses  a  test  target  that  has  a  random  multiscale  wavelet  pattern  distribution.  It 
allows  a  statistical  averaged  measurement  of  the  camera  wavelet  amplitude  reduction  according  to  the 
wavelet  scale,  thus  providing  a  new  figure  of  merit. 

The  second  method  consists  in  setting  the  measurement  free  from  the  aliasing  effects.  Its  aim  is  to 
measure  the  system  transfer  response  (base  band)  and  the  spurious  response  replicas  (using  the  NVTherm 
keywords).  This  method  uses  a  steady  distribution  of  wavelet  patterns  and  is  based  on  the  shape  distortion 
analyse  of  the  output  wavelet  patterns  from  the  camera.  The  image  processing  analyse  procedure  is  able  to 
provide  an  automatic  measurement  technique  without  any  mechanical  adjustment  requirements. 

The  feasibility  of  both  methods  was  proved  by  using  a  computer  simulation  of  a  modelled  camera 
measurement.  However  the  main  problem  remains  being  the  manufacturing  of  grey  level  test  target. 


2.7.1.  Performance  assessment  with  a  random  multi-element  target 

The  test  target  for  this  first  assessment  method,  Fig.  2.10,  is  composed  of  a  set  of  wavelet  patterns  that  are 
spatial  frequency  band  limited.  The  patterns  are  distributed  according  to  a  fractal  algorithm  using  random 
variables,  in  such  a  way  that  location  of  several  pattern  scales  are  randomly  distributed  on  the  target 
surface,  without  spatial  overlapping  between  elements.  Since  the  pattern  cut-off  frequency  (Fw)  is 
inversely  proportional  to  the  pattern  scale,  the  several  scale  pattern  set  will  provide  a  gradual  evaluation  of 
the  aliasing  phenomenon. 
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Fig.  2.10  Random  multi-element  target  with  band  limited  wavelet  pattern  (horizontal). 


The  statistical  analysis  of  the  camera  wavelet  amplitude  reduction  according  to  the  wavelet  scale 
gives  a  new  figure  of  merit  describing  the  average  amplitude  reduction  yielded  by  aliasing  and  non  perfect 
reconstruction  of  the  camera  (Fig.  2.1 1). 


2.7.2.  Performance  assessment  with  a  steady  position  multi-element  target 

The  second  resolution  measurement  method  is  based  on  steady  position  wavelet  patterns  with  two  pattern 
scales,  Fig.  2.12.  Large  elements  are  dedicated  to  the  small  elements  localisation  in  the  image  produced  by 
the  imager.  The  small  elements  responses  are  used  for  resolution  assessment. 

The  calculation  method  uses  some  principles  of  the  GLSF  MTF  measurement  technique  (see 
Section  2.6).  It  consists  in  finding  the  distortion  periodicity  number  Kw  of  the  small  wavelet  patterns 
aligned  in  columns,  Fig.  2.13.  The  Kw  extracted  patterns  horizontal  luminance  allows  the  calculation  of  a 
spectral  representation  of  the  sampling  and  reconstruction  camera  process.  This  figure  of  merit  gives  the 
system  transfer  response  (pre-post  filter  response)  and  the  spurious  response  replicas  (post-filtered  pre¬ 
filtered  replicated  responses).  Fig.  2.14  shows  an  example  of  a  simulated  measurement,  for  an 
undersampled  camera  with  a  LED  display,  together  with  an  image  example. 


Fig.  2. 1 1 :  Random  multi-element  target  with  band  limited  wavelet  pattern  (horizontal). 
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Fig.  2.12:  Multi  elements  steady  position  target  with  band  limited  wavelet  patterns  (horizontal) 


Fig.  2.13:  Periodical  wavelet  distortion,  and  extraction  of  Kw  patterns  (here  Kw=5). 
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Fig.  2.14:  System  transfer  response  (base  band)  and  the  spurious  response  replicas. 


2.8  AMOP 

AMOP  (Average  Modulation  at  Optimum  Phase)  is  a  performance  measure  used  in  the  TRM3  model.  It  is 
the  average  signal  difference  in  the  image  of  the  4-bar  standard  pattern,  with  the  test  pattern  positioned  at 
optimum  phase.  It  can  be  measured  in  the  video  signal  or  on  the  display. 
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In  case  of  well-sampled  imagers,  the  signal  difference  between  individual  bars  of  the  reconstructed 
test  pattern  is  constant  (assuming  no  noise)  and  independent  of  the  position  of  the  test  pattern,  see  Fig. 
2.15.  (Classical  performance  models  assume  that  this  average  signal  is  proportional  to  4/71  times  the 
system  MTF.) 


Fig.  2. 15:  Reconstructed  signals  of  a  standard  4-bar  test  pattern  for  a  well-sampled  low-pass 
filtered  imager 


For  undersampled  imagers  the  modulation  between  individual  bars  is  not  constant  but  depends  on 
the  position  or  phase  of  the  test  pattern  relative  to  the  detector  matrix.  An  example  is  given  in  Fig.  2.16. 
With  the  nomenclature  of  Fig.  2.17,  the  average  signal  difference  or  ‘bar  strength’  at  phase  cp  is  given  by 

AS(r,cp)  =  ^-r-X|Si-Si+1| 
n-l  tt 

It  was  shown  that  there  is  an  optimum  phase  position  at  which  the  average  signal  difference  is  a 
maximum. 1 1  This  average  value  is  designated  as  AMOP. 


opt 


Fig.  2.16:  Sampling  of  a  4-bar  target  having  a  spatial  frequency  near  the  Nyquist  frequency  and 
moving  over  a  staling  array.  The  reconstructed  signal  depends  on  the  phase.  The 
optimum  phase  is  on  the  top  right  side  of  the  figure. 


Fig.  2.17:  Example  of  the  signal  variation  in  a  sampled  and  reconstructed  standard  4-bar  target  in 
case  of  an  undersampled  imager. 
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Fig.  2.18  gives  an  example  of  AMOP  as  function  of  spatial  frequency,  normalised  with  the 
sampling  frequency.  The  prefilter  MTF  is  also  shown  for  comparison.  AMOP  takes  values  lower  than  4/71 
times  the  prefilter  MTF  and  drops  to  zero  at  about  0.9  times  the  sampling  frequency  of  the  imager.  For 
details  see  the  bibliography. 


Fig.  2.18:  Example  of  AMOP  as  function  of  the  spatial  frequency,  normalized  with  the  sampling 
frequency.  AMOP  takes  values  lower  than  4/71  times  the  prefilter  MTF. 


2.9  DOUBLE-SLIT  METHOD 

The  double-slit  method12  uses  two  parallel  line  sources  the  distance  of  which  can  be  varied.  In  general,  the 
modulation  depth,  defined  as  the  ratio  of  the  signal  dip  Vd  between  both  signal  peaks  and  the  peak  signal 
vm,  is  measured  as  function  of  the  angular  separations  between  the  two  line  sources,  see  Fig.  2.19.  The 
position  (phase)  of  the  double  slit  relative  to  the  detector  grid  is  optimised  for  maximum  modulation 
depth. 


Fig.  2. 19:  Example  of  a  signal  obtained  with  the  double-slit  technique. 


Fig.  2.20  shows  the  results  of  the  measurements  on  the  modulation  depths,  carried  out  on  a  number 
of  cameras.  One  may  observe  the  strong  dependence  of  the  modulation  depth  with  the  slit  spacing. 
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Fig.  2.20:  Modulation  depths  measured  with  the  double-slit  technique. 


The  resolution  in  mrad  1  is  defined  as  the  inverse  distance  for  which  the  modulation  depth  is  10%. 
In  general,  the  lines  can  be  oriented  in  any  direction:  Measurements  in  the  video  signal  are  made  for 
vertical  double-slits  where  improvement  of  the  signal  to  noise  ratio  is  achieved  by  frame  averaging. 
Measurements  are  obtained  from  the  display  with  an  external  high-resolution  video  camera. 

The  advantage  of  the  double-slit  method  is  the  ability  to  reach  a  fine  adjustment  of  the  focus  and  the 
definition  of  resolution  similar  to  the  classical  Rayleigh  criterion.  The  drawback  is  the  non-direct 
relationship  of  the  modulation  depth  with  the  classical  MTF. 


This  page  has  been  deliberately  left  blank 
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Chapter  3 

Assessment  of  System  Sensitivity 


3.1  CLASSICAL  FIGURES  OF  MERIT 

Signal  Transfer  Function  (SiTF)  and  Noise  Equivalent  Temperature  Difference  (NETD  are  the  main 
figures  of  merit  to  assess  dynamic  range  and  thermal  sensitivity  of  1st  generation  scanning  imagers. 

The  SiTF  is  the  output  signal  (voltage  or  luminance)  as  function  of  the  input  signal  (irradiance  or 
temperature).  This  function  is  generally  S -shaped  with  a  linear  region  and  saturation  regions  at  low  and 
high  irradiances  or  temperatures.  NETD  is  used  to  describe  the  temporal  noise  of  the  imager.  It  is  defined 
as  the  temperature  difference  between  an  extended  test  target  and  the  background  that  produces  a  signal 
equivalent  to  the  temporal  noise.  The  classical  definition  of  NETD  is  referred  to  an  electronic  standard 
reference  filter.  It  is  measured  in  the  analog  output  signal. 

3.2  NOISE  CHARACTERISATION  OF  ADVANCED  SYSTEMS 

Modern  imaging  sensors  incorporating  focal  plane  architectures  have  sophisticated  post-detector 
processing  and  electronics.  These  advanced  technical  characteristics  produces  the  generation  of  complex 
noise  patterns  in  the  output  of  the  system.  Unlike  classical  systems  where  “well-behaved”  detector  noise 
predominates,  these  complex  systems  have  the  ability  to  generate  a  wide  variety  of  noise  types  each  with 
distinctive  characteristics  temporally  as  well  as  along  the  vertical  and  horizontal  image  directions. 

Such  complex  three-dimensional  (time,  vertical,  horizontal)  noise  phenomena  cannot  be 
adequately  treated  by  previous  mathematical  analyses  developed  for  earlier  simpler  systems.  Different 
techniques  and  assessment  parameters  were  developed  to  characterise  the  noise  in  advanced  systems. 
Also,  detector  elements  with  low  responsivity,  excessive  noise  or  1/f  noise  (bad  pixels )  must  be  identified, 
characterised  and,  if  necessary,  excluded  from  the  measurements  since  they  affect  the  measurement 
results. 

3.2.1.  General  considerations 

In  advanced  systems,  noise  measurements  can  be  made  at  different  output  ports  of  the  imager  (digital  or 
analog  video)  and  in  specified  regions  of  interest.  Generally,  imager  noise  is  assessed  without  using  any 
filters.  Bad  pixels  may  or  may  not  have  been  excluded  from  measurements.  The  results  of  noise 
measurements  may  strongly  depend  on  the  test  conditions  that  must  be  reported  when  presenting  data. 

3.2.2.  Temporal  noise 

Temporal  noise  can  be  assessed  either  separately  for  each  detector  by  analysing  the  temporal  variation  of 
the  detector  signal,  or  as  an  average  noise  over  many  detectors.  TG12  recommends  using  the  term  NETD 
only  to  characterise  temporal  noise.  NETD  should  not  be  used  for  the  joint  assessment  of  spatial  and 
temporal  noise. 

As  already  mentioned,  temporal  noise  measurements  shall  not  be  performed  with  the  classical 
standard  reference  filter  or  any  other  filter  that  limits  the  bandwidth  of  the  video  signal. 

3.2.3.  Spatial  noise 

Individual  detector  elements  in  infrared  detector  arrays  have  different  photo  response  characteristics. 
Unless  corrected,  the  displayed  image  is  not  uniform,  even  if  a  homogeneous  scene  at  constant 
temperature  is  viewed:  The  detector  non-uniformity  appears  as  stationary  or  fixed  pattern  noise  in  the 
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image.  Non-uniformity  can  vary  with  the  type  of  technology,  detector  material  and  fabrication  technique 
employed  in  the  manufacture  of  a  particular  FPA  as  well  as  on  post-detector  processing  and  electronics. 

High  performance  FPAs  need  to  be  able  to  resolve  thermal  image  scenes  with  a  temperature 
difference  typically  as  small  as  0.01  K,  thus  requiring  the  spatial  non-uniformity  to  be  in  the  same  order  of 
magnitude.  Therefore,  non-uniformity  correction  is  a  must. 

3.2.4.  Non-uniformity  correction 

The  detector  non-uniformity  is  corrected  by  measuring  the  detector  signal  responses  at  one,  two  or  more 
uniform  blackbody  temperatures.  The  measured  offset  and  (in  case  of  a  two-point  correction)  gain 
differences  of  each  detector  signal  relative  to  the  average  offset  and  gain  are  used  to  calculate  the  required 
offset  (and  gain)  correction.  The  correction  is  then  applied  to  each  detector  signal  with  the  aim  to  obtain 
uniform  response. 

The  correction  is  never  perfect,  thus  producing  residual  non-uniformity  in  the  displayed  image 
that  needs  to  be  assessed.  Best  correction  is  obtained  at  the  calibration  temperatures  immediately  after 
determining  the  correction  coefficients.  At  other  background  temperatures  the  residual  non-uniformity  is 
higher.  An  example  is  shown  in  Fig.  3.1  for  a  system  with  two-point  correction. 


Fig  3.1:  Measured  spatial  non-uniformity  as  function  of  the  background  temperature  for  an 
imager  with  two-point  correction.  (Calibration  temperatures  Ti  and  T2).  Non-uniformity 
is  not  equal  to  zero  at  the  calibration  temperatures. 


Residual  non-uniformity  also  changes  with  time.  This  is  due  to  the  temporal  instability  of  the  detector 
elements  in  the  array,  which  depends  on  detector  type,  detector  manufacturing  technology,  cooler 
instability,  etc. 


3.2.5.  Real-time  non-uniformity  correction 

Real-time  non-uniformity  correction  (NUC)  methods  serve  to  always  keep  the  residual  non-uniformity  at  a 
minimum  level.  The  assessment  of  real-time  NUC  techniques  is  an  important  issue,  which,  however,  was 
not  addressed  by  TGI 2. 

3.3  3-D  NOISE  MODEL 

The  3-D  noise  model13,14  divides  the  total  noise  present  into  a  set  of  eight  components  which  have  special 
properties  related  to  temporal  and  spatial  dimensions  which  form  a  three-dimensional  coordinate  system. 
One  axis  is  time.  The  other  two  are  horizontal  and  vertical  dimension.  When  a  number  of  frames  are  taken 
and  recorded  it  is  possible  to  obtain  eight  kind  of  data  set,  each  of  one  corresponding  to  a  different  type  of 
noise.  These  sets  of  data,  Nx,  are  formed  form  the  original  set  of  frames  by  means  of  averaging  operators. 
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The  exact  procedure  to  obtain  these  sets  is  complicate  and  too  extensive  to  explain  in  this  document,  so 
the  reader  is  invited  to  consult  the  bibliography. 


Ntvh  Contains  tvh  noise 

Nvh  Contains  vh  noise  only 

Nth  Contains  th  noise  only 

Ntv  Contains  tv  noise  only 

Nv  Contains  v  noise  only 

Nh  Contains  h  noise  only 

Nt  Contains  t  noise  only 


It  is  possible  to  obtain  information  about  noise  from  this  data  set.  For  example,  a  figure  of  merit 
similar  to  IETD  (see  next  paragraph)  can  be  derived  from  the  following  expression, 

<Jvh  =  std(  Nvh(  1  -  F  ))/ R  . 

being  F  an  operator  that  removes  low  frequency  components  from  Nvh  and  “std”  means  standard  deviation, 
R  is  the  slope  of  the  SiTF  at  measurement  temperature.  Similarly  NETD  can  be  compared  with, 

vh  =std(Ntvh(l-F  ))/R. 


Other  noise  descriptors  can  be  defined  in  a  similar  way.  The  interest  in  this  procedure  is  that  it  is 
similar  to  three  factor  components  of  a  variance  analysis  in  which  temporal  (frame-to-frame),  vertical 
(row-to-row),  and  horizontal  (column-to-column)  effects  arc  partitioned  from  the  total  noise  present  in  the 
data  set.  It  permits  a  characterization  of  different  kinds  of  noise  what  will  be  important  in  follow  sections 
(MRTD,  MTDP).  It  is  valid,  too,  for  scanning  and  staling  systems.  For  scanning  systems  with  analog 
output  it  is  necessary  to  digitise  the  image  before  applying  the  method. 


3.4  IETD 

The  IETD15  (Inhomogeneity  Equivalent  Temperature  Difference)  describes  the  RMS-variation  of 
randomly  distributed  fixed  pattern  noise  in  terms  of  temperature,  similar  to  NETD,  which  describes  the 
temporal  high  frequency  noise  of  the  imager. 

IETD  is  defined  as: 

IETD(  T )  =  — . 

dV/dT 

where  <7V  is  the  RMS  value  of  the  randomly  distributed  fixed  pattern  noise  voltage  and  dV/dT  is  the 
responsivity  of  the  system. 

To  calculate  IETD,  low  frequency  contributions  and  periodic  artefacts  (stationary  or  pseudo¬ 
stationary  line  or  column  structures)  are  removed  by  high-pass  filtering  and  clipping  of  excessive 
frequency  components  from  the  measured  fixed  pattern  noise  signal.  An  example  is  given  in  Fig.  3.2 
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Fig  3.2:  Thermal  image  after  two-point  correction  (top  left)  and  signal  variation  in  a  horizontal 
line  (bottom  left).  The  same  image  after  high-pass  filtering  (top  right)  and  line  signal 
after  high-pass  filtering  (bottom  right). 

The  actual  fixed  pattern  noise  in  an  advanced  imager  may  be  more  complex  due  to  spatial  and 
spatial-temporal  artefacts  introduced  by  imperfect  electronic  circuits  (periodic  or  pseudo-periodic  line 
patterns,  low  frequency  variations,  flicker,  etc).  IETD  assesses  only  that  contribution  of  spatial  and 
temporal-temporal  noise  whose  impact  on  system  performance  can  be  modelled  analytically. 


3.5  CORRECTABILITY 

This  figure  of  merit16  provides  an  estimate  of  the  residual  spatial  noise  after  calibration  correction  of  the 
camera,  relative  to  the  temporal  noise.  A  correctability  value  of  one  means  that  spatial  noise  after 
correction  is  equal  to  temporal  noise.  A  value  smaller  than  one  means  that  spatial  noise  is  lower  than 
temporal  noise. 

The  correctability  figure  of  merit  is  based  on  the  statistical  analysis  of  noise  data  taken  at  various 
background  temperatures.  The  statistics  is  described  as  follows 

2_f  (yfr<yi>)2 

A  j  2 

i=i  yti 


where  yf  is  the  value  of  the  pixel  j  after  correction  at  temperature  i ,  ^y(.y  the  mean  signal  of  the  array  at 

temperature  i  and  yti  the  temporal  noise  of  the  array  at  temperature  i.  The  X  distribution  for  a  FPA  with 
uniform  response  and  randomly  distributed  Gaussian  temporal  noise  is  well  known  in  statistics. 

Deviations  of  the  measured  ^“histogram  from  the  ideal  distribution,  see  Fig.  3.3,  are  due  to  the 
residual  spatial  non- uniformity  in  the  array.  After  subtracting  the  ideal  distribution  due  to  temporal  noise 
from  the  real  distribution  after  correction,  one  obtains  a  measure  to  estimate  the  quality  of  the  array.  This 
measure  is  defined  the  correctability  of  the  focal  plane  array.  It  is  given  by 


c  = 


2>; 


j= i 


(N  -  \){n  -  r) 


■1  = 


yf(y^~<yi>) 


j= 1  i= 1 
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where  N  is  the  number  of  pixels  in  each  frame  of  data  taken, 

i.&G-O 


0  i  8  12  16  2C  24  28  32  16 


Fig  3.3:  Probability  Distribution  Flistogram  and  theoretical  distribution. 


3.6  THE  PRINCIPAL  COMPONENT  APPROACH 

A  different  approach  of  characterizing  thermal  imager  noise  is  by  performing  a  principal  component 
decomposition.17  Its  general  objectives  are  data  reduction  and  interpretation  of  noise  data. 

The  set  of  noisy  frames  provided  by  a  thermal  imager  with  M  detectors  is  represented  as  an  N  x  M 
matrix  F  with 


F  =  {Fj,  F2,  ....,  Ft, . Fn,}  , 

where  N  is  the  number  of  frames  and  Ft  is  the  frame  taken  at  a  given  time.  F  contains  the  full  set  of  data  to 
be  analysed.  Each  frame  can  be  considered  a  random  variable.  The  realisations  of  this  variable  are  the 
signals  obtained  by  each  detector  at  the  given  frame.  To  demonstrate  the  approach.  Fig.  3.4  gives  an 
example  with  three  frames. 


Fig  3.4:  Scatter  plot  of  the  pixel  values  in  the  three  frames  indicating  the  principal  components 


Each  detector  is  represented  as  a  point  whose  coordinates  are  the  three  different  signal  values  of  that 
detector  in  the  three  frames.  When  all  the  pixels  are  located  in  this  diagram  a  cloud  of  dots  is  obtained.  In 
Fig.  3.4  the  pixels  are  mainly  aligned  along  a  given  direction.  This  direction  corresponds  to  the  first 
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principal  component  which  contributes  most  to  the  variance  of  the  data.  The  second  and  third  principal 
components  are  located  in  planes  perpendicular  to  the  first  one. 

The  principal  component  decomposition  takes  this  set  of  image  frames  as  a  linear  combination  of 
eigen-images,  each  one  associated  with  one  principal  component,  see  Fig.  3.5.  When  applying  the  method 
to  the  analysis  of  noisy  images,  it  allows  to  identify  the  different  noise  processes  involved  in  the  noise  and 
to  calculate  the  contribution  of  these  noise  processes  to  the  total  noise  variance. 


Fig  3.5:  Eigen-images  of  the  first  four  principal  components  in  a  set  of  noise  frames.  The  first 
eigen-image  is  associated  with  stationary  fixed  pattern  noise.  The  second  and  third 
eigen-image  is  associated  with  a  fringe  pattern  crossing  the  scene  periodically.  The 
fourth  is  associated  with  temporal  noise. 

The  correlation  coefficients  defined  by  Mooney18  can  be  calculated  by  means  of  the  covariance 
matrix  for  the  different  noise  processes.  Also,  the  3-D  methodology  can  be  applied  to  each  noise  process 
separately.  The  results  of  the  method  can  be  presented  as  images.  Therefore,  the  analysis  of  spatial 
patterns  associated  with  each  type  of  noise  process  is  easily  derived.  The  technique  allows  a  very  detailed 
assessment  of  the  imager  noise,  which  cannot  be  accomplished  by  previous  methods. 


3.7  BAD  PIXEL  CHARACTERIZATION 


3.7.1.  Phenomenology  and  definitions 

RSG16  of  NATO/AC243  adopted  the  following  definitions.19 


■  Bad  pixel:  Those  detector  elements  (pixels)  in  a  focal  plane  array  which  are  either  permanently 

dead  or  which  show  abnormal  characteristics  are  called  bad  pixels. 

■  Dead  pixel:  A  detector  element  is  a  dead  pixel  if  it  has  no  response  at  all;  i.e. 


AS  dT) 

R:  =  — —  =  0 
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■  Abnormal  pixel:  A  detector  element  is  defined  as  abnormal  if  one  of  the  two  following  conditions 
are  met: 


o  excessive  temporal  noise,  i.e.  NETD  >  NETDtoierabie 
o  correctability  is  above  a  specified  threshold 


Abnormal  pixels  have  one  or  several  of  the  following  features: 

•  Excessive  temporal  noise 

•  Detector  response  beyond  tolerable  limits 

•  Detector  offset  beyond  tolerable  limits 

•  Unacceptable  temporal  signal  variations  (1/f  noise;  deviant  flashers;  blinker) 


23 


The  general  assessment  of  bad  pixels  is  addressed  in  the  next  two  paragraphs.  The  assessment  of 
unacceptable  temporal  signal  variations  is  heated  in  Sections  3.74  and  3.75. 

3.7.2.  Bad  pixel  assessment  with  the  correctability  concept 

The  correctability  concept  can  be  used  to  characterize  any  abnormal  feature  of  a  pixel,  except  for 
excessive  noise.  A  pixel  can  be  characterized  as  a  bad  pixel  if  its  X]  value  exceeds  the  value  at  which  the 

ideal  X  distribution  of  only  temporal  noise  is  zero  (see  Fig.  3.3).  The  concept  permits  to  specify  a 
reasonable  threshold  to  eliminate  bad  pixels  in  a  focal  plane. 

The  correlation  between  bad  pixels  as  defined  by  an  excessive  NETD  and  by  the  correctability 
procedure  was  investigated.20 


3.7.3.  Bad  pixel  assessment  with  the  principal  component  approach 

It  is  possible  to  define  an  average  or  mean  pixel  as  a  pixel  that  exhibits  the  mean  behaviour  with  respect  to 
the  principal  components.  Its  components  are  the  mean  components  of  all  pixels  with  respect  to  each 
principal  component.  A  distance  measure  is  introduced  to  characterise  the  distance  between  each 
individual  pixel  and  the  mean  or  average  pixel.21  This  is  illustrated  in  Fig.  3.6.  Different  distances 
measures  can  be  introduced  depending  on  the  required  data  analysis. 

A  detector  element  is  considered  being  a  bad  pixel  if  its  distance  from  the  average  pixel  exceeds  a 
specified  threshold  value. 


d 


Fig.  3.6: 


View  of  the  detectors  in  the  principal  component  coordinate  system.  Some  pixels  can 
easily  be  identified  as  bad  pixels. 


3.7.4.  1/f  noise,  long  term  stability 

Detector  elements  with  unacceptable  temporal  signal  behaviour,  see  Fig.  3.7,  were  designated22  as  either 

•  Weak  pixel:  Detector  elements  showing  a  weak  sensitivity.  The  corrections  procedure  strongly 
amplifies  the  noise  together  with  the  signal 

•  Blinker  pixels:  Detector  elements  characterized  by  a  burst  noise  switching  between  two  or  more 
states.  They  give  a  high  contribution  to  the  1/f  noise 

•  Drifting  pixels:  Elements  who  exhibit  a  slow  drift  of  its  mean  signal  value.  They  also  give  a 
great  contribution  to  the  1/f  noise. 
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Fig  3.7:  Temporal  signal  variations  of  a  blinker  pixel  (left)  and  of  a  drifting  pixel  (right).  For 
comparison,  the  temporal  signal  variation  of  a  regular  detector  element  is  also  shown  in 
the  two  graphs. 


Unacceptable  temporal  signal  behaviour  is  characterized  as  1/f  noise.  1/f  noise  is  present  in  many  of 
FPA  imagers.  It  is  detected  by  analysing  the  noise  power  spectrum  of  individual  detector  elements.  The 
1/f  slope  in  the  spectrum  can  easily  be  identified  in  a  log-log  plot.  Normally,  this  kind  of  noise  enhances 
low  temporal  frequencies  compared  to  high  temporal  frequencies.  It  represents  a  low  frequency  trend  in 
the  detector  signal.  To  detect  and  measure  1/f  noise  it  is  necessary  to  record  frames  during  long  periods  of 
time  (normally  minutes  and  hours).  Fig.  3.8  shows  as  an  example  the  signal  of  a  detector  with  1/f  noise 
and  its  noise  power  spectrum. 


frequency 


Fig  3.8:  Typical  signal  variation  in  a  detector  element  with  1/f  noise  (left)  and  associated  noise 
power  spectrum  (right).  Units  are  arbitrary. 


3.7.5.  1/f  noise  assessment  with  the  correctability 

The  correctability  approach  allows  the  assessment  of  1/f  noise23.  1/f  noise  affects  correctability  in  two 
ways: 

•  The  maximum  of  the  X  histogram  is  moved  toward  lower  values. 

2 

•  The  tail  of  the  X  histogram  becomes  longer;  an  indicator  for  a  large  number  of  non-correctable 
pixels. 

2 

From  the  X  histogram  the  1/f  coefficient,  Cf,  is  derived  which  is  given  by 
c f  =  (a2  —  <J2  )/ <J2 

j  \  ms  mx '  mx 
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where 


r2 

A  max 

n  -3 


and 


<z2> 

7 1  -1 


2  2 

n  is  the  number  of  frames  used  for  the  analysis  of  the  correct  ability,  Xmax  the  maximum  X  value  and 
<  X  >  the  mean  square  of  the  histogram.  If  the  arrays  has  no  pixels  with  1/f  noise  cf  is  zero.  In  the 
presence  of  1/f  noise  the  histogram  is  deformed  and  c  f  increases  with  the  “amount”  of  1/f  noise  present. 
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Overall  Performance  Measures 
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4.1  INTRODUCTION 

The  figures  of  merit  outlined  in  previous  sections  characterize  the  imager  in  terms  of  resolution  and 
sensitivity.  In  this  section  overall  device  performance  characterization  is  addressed. 

Figures  of  merit  that  assess  the  overall  device  performance  must  include  the  thermal  sensitivity  of 
the  imager,  its  spatial  resolution  including  the  sampling  of  information,  and  observer  performance.  The 
classical  MRTD,  which  is  shortly  reviewed  in  the  next  paragraph,  is  not  suited  to  assess  undersampled 
advanced  imagers.  New  techniques  were  developed  which  are  described  in  the  remaining  sections  of  this 
section. 

Human  observers  determine  the  figures  of  merit  described  in  Sections  4.2  to  4.6.  The  results  obtained 
with  these  measures  are  of  subjective  nature.  Several  objective  techniques  were  developed  that  are 
described  in  Sections  4.7  and  4.8.  These  techniques  replace  the  human  observer  by  an  eye-brain  model. 


4.1.1.  MRTD  and  undersampling 

MRTD  (Minimum  Resolvable  Temperature  Difference)  is  the  classical  figure  of  merit  developed  for  1st 
generation  imagers.  It  is  the  minimum  temperature  difference  at  which  the  four  bars  of  the  standard  4-bar 
test  pattern  are  resolved  by  an  observer.  The  measurement  procedure  is  standardized  in  STANAG  4349.24 

If  the  imager  is  undersampled,  the  image  of  the  4-bar  test  pattern  depends  on  the  position  of  the  test 
pattern  relative  to  the  detector  array  (phase).  The  four  bars  can  be  resolved  only  at  pattern  frequencies 
lower  than  about  0.55  times  the  sampling  frequency.  At  higher  frequencies  the  four  bars  cannot  be 
resolved  due  to  aliasing.  This  is  illustrated  in  Fig.  4.1. 


Fig.  4. 1 :  Images  of  standard  4-bar  test  patterns  obtained  with  a  well-sampled  imager  (left)  and  an 
undersampled  one  (right). 

If  undersampled,  the  four  bars  can  be  perceived  only  at  pattern  frequencies  lower  than 
approximately  the  Nyquist  frequency.  Shape  and  modulation  of  the  displayed  bar 
pattern  depends  on  the  phase  (see  the  encircled  pattern).  At  frequencies  greater  than  the 
Nyquist  frequency  the  four  bars  cannot  be  resolved. 
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4.2  STATIC  MRTD 

Figure  4.2  shows  the  static  approach  to  measuring  MRT  of  a  staring  sensor.  At  each  frequency  or  bar 
pattern,  the  phase  is  found  which  optimises  the  MRT.  To  vary  the  phase,  the  target  wheel  is  placed  on  a 
translating  stage,  or,  equivalently,  the  sensor  is  placed  on  a  motorized  rotational  stage.  The  phase  is 
considered  the  angular  target  position  with  respect  to  the  sensor  detector.  While  the  observer  views  the  4- 
bar  target,  the  stage  is  adjusted  (i.e.  the  phase  is  varied)  or  the  sensor  is  micro-positioned  to  peak  up  the  4- 
bar  target  modulation.  Between  0.6  and  0.9  times  the  half-sample  rate  of  the  sensor,  it  is  extremely 
difficult  to  ensure  that  correct  target  phase  is  accomplished.  The  results  vary  greatly  with  phase  and  from 
observer  to  observer.  Figure  4.2  on  the  right  side  shows  two  images  of  a  4-bar  target  where  one  target  is 
positioned  at  best  phase  and  the  other  target  is  positioned  at  worst  phase.  Figure  4.3  shows  the  MRT  for  a 
sensor  at  best  phase  and  at  worst  phase.  Also  note  in  the  static  MRT  Figure,  the  MRT  is  measured  past 
the  half-sample  rate  of  the  sensor  for  the  best  phase  case. 25 


Blackbody 


Best  Phase 


Worst  Phase 


Fig.  4.2:  Sample  imaging  system  MRTD  test  configuration  (left)  and  images  of  the  standard  4-bar 
target  at  best  and  worst  phase  (right). 


4.3  DYNAMIC  MRTD 

In  order  to  address  the  phase  and  sampling  problems  associated  with  the  MRT  measurement,  Webb  26 
developed  the  dynamic  MRT  measurement  where  the  MRT  target  is  moved  across  the  sensor  field  of  view 
during  the  MRT  measurement.  The  technique  is  known  as  the  Dynamic  MRT  (DMRT)  and  the  MRT  is 
measured  as  the  target  is  moved  through  the  sensor's  field-of-view. 
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Fig.  4.3:  Static  MRTD  measurements  at  different  phases  (left)  and  comparison  of  static  and 
dynamic  MRTD  measurements  (right). 
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It  has  been  shown  that  the  DMRT  can  measure  system  performance  up  to  the  MTF  limit  of  the 
system  and  eliminates  the  problems  associated  with  static  sampling  and  phase  optimisation.  The  increase 
in  MRT  performance  is  measured  due  to  an  increase  in  the  sample  rate  across  the  target  (as  the  target 
moves  across  the  sampling  grid). 

There  are  two  characteristics  here  worth  mentioning.  First,  while  the  sampling  effect  has  been 
minimized  in  the  DMRT  measurement,  there  has  been  an  additional  MTF  blur  introduced  by  the  motion  of 
the  target.  The  DMRT  measurement  changes  with  target  velocity  through  the  sensor  field-of-view  since 
both  the  spatial  sampling  rate  has  changed  and  the  blur  due  to  motion  has  changed.  The  DMRT 
measurement  beyond  the  half-sample  rate  of  the  sensor  is  useful  and  can  be  modified  for  use  in  acquisition 
calculations.  Webb  and  Halford  describe  the  DMRT  method  in  detail. 27 

4.4  MTDP 

The  MTDP28  (Minimum  Temperature  Difference  Perceived)  allows  assessment  of  both  well-sampled  and 
undersampled  imagers.  The  MTDP  figure  of  merit  is  used  in  the  German  TRM3  model.  It  is  based  on  the 
perception  of  the  standard  4-bar  test  pattern.  The  MTDP  concept  accepts  that  in  case  of  undersampling  the 
image  of  the  standard  test  pattern  is  distorted  and  that  less  than  four  bars  are  resolvable  at  pattern 
frequencies  greater  than  about  half  the  sampling  frequency. 

MTDP  is  defined  as  the  minimum  temperature  difference  at  which  the  four,  three  or  two  bars  of  the 
(distorted)  test  pattern  are  resolved  by  an  observer.  It  is  defined  for  and  measured  at  optimum  phase  (see 
Section  2.8).  For  a  well-sampled  imager  each  phase  is  optimum  and  hence  MTDP  and  MRTD  become  the 
same. 


To  measure  MTDP,  first  the  optimum  phase  has  to  be  found  at  a  sufficiently  high  AT.  Once  the 
pattern  is  positioned  at  optimum  phase,  the  same  procedures  are  used  as  in  MRTD  measurements.  MTDP 
is  related  with  AMOP  (see  Section  2.8)  as  follows: 


MTDP(r)  = 


nll-SNRthXl2<f) 

AMOP(r) 


where  SNRthr  is  a  threshold  signal-noise-ration  and  T  a  noise  term.  MTDP  measurements  can  be  made 
up  to  spatial  frequencies  of  approximately  1.8  times  the  Nyquist  frequency.  The  limiting  frequency 
depends  on  the  amount  of  undersampling,  see  Fig.  4.4.  In  Fig.  4.5  a  measured  MTDP  is  shown  and 
compared  with  the  prediction  made  with  TRM3. 


normalised  spatial  frequency 


Fig.  4.4:  MTDP  and  sampling.  MTDP  as  function  of  spatial  frequency  normalized  with  the 
sampling  frequency.  Undersampling  is  characterized  by  the  prefilter  MTF  at  Nyquist. 
For  a  well-sampled  imager  (MTF  at  Nyquist  =  0),  MTDP  and  MRTD  are  the  same. 
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Fig.  4.5:  Example  of  a  measured  MTDP  (circles)  and  comparison  with  its  prediction  by  TRM3 


4.5  TOD  (TRIANGLE  ORIENTATION  DISCRIMINATION  THRESHOLD) 

Another  approach  is  the  TOD  (Triangle  Orientation  Discrimination  threshold)  methodology.  29,30  The 
TOD  deals  with  the  half-sample  rate  limit  problem  associated  with  the  MRT/FLIR92  method  by  using  an 
alternative,  non-periodic  test  pattern:  an  equilateral  triangle  in  four  possible  orientations  (apex  Up,  Down, 
Left  or  Right,  see  Fig.  4.6,  panel  A). 

At  the  same  time,  the  laboratory  measurement  uses  a  robust  4AFC  (Four  Alternative  Forced-Choice) 
psychophysical  procedure  instead  of  the  subjective  MRT  measurement  procedure.  In  this  procedure,  the 
observer  has  to  indicate  which  triangle  orientation  he  sees,  even  if  he  is  not  sure.  Variation  of  triangle 
contrast  and/or  size  leads  to  a  variation  in  the  percentage  correct  between  25%  (complete  guess)  and 
100%,  and  by  interpolation  the  exact  75%  correct  threshold  can  be  obtained  (see  panel  B).  A  complete 
TOD  curve  (comparable  to  an  MRT  curve)  is  obtained  by  plotting  the  contrast  thresholds  as  a  function  of 
the  reciprocal  of  the  triangle  angular  size  (see  panel  C).  A  detailed  description  of  the  measurement  of  a 
TOD  curve  is  given  in  [30]. 

The  TOD  method  has  a  large  number  of  theoretical  and  practical  advantages:  it  is  suitable  for 
under-sampled  and  well-sampled  electro-optical  and  optical  imaging  systems  in  both  the  thermal  and 
visual  domains,  it  has  a  close  relationship  to  real  target  acquisition,  and  the  observer  task  is  easy.  The 
results  are  free  from  observer  bias  and  allow  statistical  significance  tests.  The  laboratory  method  may  be 
implemented  in  current  MRT  test  equipment  with  little  effort,  and  the  TOD  curve  can  be  used  easily  in  a 
TA  model  such  as  ACQUIRE.  In  addition,  the  method  lends  itself  very  well  for  automatic  measurement 
using  a  human  observer  model.31,32 

Three  validation  studies  with  real  targets,  including  the  simulation  experiment  described  in  the 
NATO  TG.  12  Topic  1  Final  Report,  show  that  the  TOD  curve  predicts  TA  performance  for  under-sampled 
and  well-sampled  imaging  systems  very  well.  Currently,  a  theoretical  sensor  model  to  predict  the  TOD 
(comparable  to  NVTherm  or  TRM3)  is  under  development. 
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Right  Left 

anel  A:  The  test  pattern  is  an  equi- 
ateral  triangle  that  can  be  presented  to 
he  sensor/observer  in  4  different  ori- 
ntations.  The  observer  has  to  indicate 
hich  orientation  he  perceives, 
riangles  are  presented  in  a  range  of 
izes  and  contrasts. 


Panel  C:  The  TOD  curve  is  obtained  by 
plotting  the  contrast  thresholds  for  the  different 
Mangle  sizes  as  a  function  of  Mangle  angular 
size.  Size  is  defined  as  the  square  root  of 
Mangle  area  in  mad.  The  quantity  on  the  size 
axis  is  size1,  denoted  as  SR,  and  is  measured  in 
mrad'1.  The  data  for  the  larger  Mangles  appear 
on  the  left  side,  and  for  smaller  Mangles  it 
appears  on  the  right  side.  This  way  the  size  axis 
is  similar  to  that  of  spatial  frequency.  Since  the 
vertical  axis  is  also  the  same  as  that  of  the 
MRTD  curve,  comparison  with  the  MRTD  is 
straightforward. 


SR  (mrad'1) 


triangle  contrast  (rel.  units) 

Panel  B:  For  each  size,  the  threshold  contrast  is  determined 
by  presenting  many  triangles  and  recording  the  percentage 
of  correct  responses  for  each  contrast.  Lowest  possible 
score  is  25%,  highest  score  is  100%.  The  exact  threshold  is 
found  from  the  data  by  interpolation  using  a  Weibull 
function3. 


Fig  4.6:  The  TOD  method  in  a  nut  shell 


4.6  MRID 

This  MRID  figure  of  merit  (Minimum  Resolvable  Intensity  Difference)  uses  two  parallel  line  sources  with 
variable  distance  as  test  pattern.  The  observer  is  looking  to  a  display  and  notes  the  minimum  distance 
between  the  two  lines,  at  which  he  can  just  discriminate  them  as  two  lines.  This  is  analog  to  the  classical 
Rayleigh  criterion  that  refers  to  a  point  source  rather  than  a  line  source.  The  MRID  is  then  defined  as  the 
intensity  difference  of  the  two  lines  for  a  given  line  separation  distance,  required  by  the  observer  to  be 
able  to  discriminate  the  two  lines.  Rather  than  the  spatial  frequency,  the  inverse  of  the  separation  distance 
is  given  as  spatial  parameter. 
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There  are  a  number  of  benefits  for  this  type  of  double-line  targets  such  as 

•  the  slit  distance  is  easy  to  vary,  so  in  the  spatial  domain  one  does  not  need  a  large  number  of  bar 
patterns 

•  line  sources  are  easy  to  model;  they  do  not  have  higher  harmonics  like  the  bars 

•  modulation  depth  is  strongly  dependent  on  the  slit  separation  distance 

•  the  use  of  two  lines  excludes  the  wrong  number  counting  as  is  the  case  with  the  4  bars,  where  one 
may  observe  2  or  3  bars  beyond  the  Nyquist  frequency 

•  the  double-slit  method  can  be  used  for  objective  as  well  as  subjective  performance  measurements 

•  the  target  is  easy  to  manufacture  and  cheap 

An  example  of  imagery  of  double  line  sources  is  shown  in  Fig.  4.7  for  two  slit  distances.  In  the 
picture  on  the  left  side  the  two  slits  are  just  discernable.  For  more  details  see  ref.  [12]. 


Fig  4.7:  Images  of  the  double  slit  test  pattern  at  two  different  slit  distances.  In  the  picture  on  the 
right  side  the  two  slits  are  just  discernable 


Similar  to  the  MRTD  curve  described  before,  one  can  define  an  MRID  curve  where  the  intensity  is 
specified  for  the  line  source.  The  apparent  intensity  of  the  wire  is  calibrated  with  an  extended  blackbody 
source,  covering  an  area  containing  several  detector-elements.  MRID  can  be  used  to  predict  the  range 
performance  of  thermal  imagers  similar  to  the  approach  of  STANAG  4347. 

A  comparison  of  MRTD  (limited  at  Nyquist),  MTDP  and  MRID  is  shown  in  Fig.  4.8. 
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Fig  4.8: 


Comparison  of  MRID  with  MTDP  and  MRTD  limited  at  Nyquist. 
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4.7  MRTD  DERIVED  FROM  LSF  AND  NOISE  MEASUREMENTS 

This  ‘objective’  technique33  is  utilizing  a  target  set-up  as  shown  in  Fig.  4.9  with  two-tilted  line  sources  and 
an  area  source  with  a  fixed  temperature  difference  AT  to  its  background.  The  camera  under  test  is 
focussed  to  the  two-line  sources  Li  and  L2  by  measuring  the  video  signal  of  the  thermal  imager  and 
maximizing  this  signal.  A  high-resolution  video  camera  is  used  to  image  the  display  of  the  imager  under 
test  in  such  a  way  that  only  paid  (e.g.  1/3)  of  the  field  of  view  is  covered  in  order  to  obtain  sufficient 
oversampling.  The  gain  setting  of  both  the  imager  under  test  and  the  video  camera  is  set  in  such  a  way  that 
the  signal  of  the  extended  area  source  as  well  as  the  line  sources  stay  within  the  linear  region  of  the  Signal 
Transfer  function.  If  necessary  their  temperatures  are  reset  to  appropriate  values. 


Fig  4.9:  Target  used  in  LSF  +  Noise  Objective  MRTD  method.  Two  lines  sources  tilted  by  an 

angle  a  and  an  extended  blackbody  source. 


The  noise  is  measured  from  the  video  output  as  RMS  value  of  the  signal  bit  levels  in  the  extended 
target  area.  By  taking  the  responsivity  the  NETD  is  calculated.  Horizontal  and  vertical  MTFs  are 
calculated  from  the  Left’s  for  line  sources  Li  and  L2.  Finally,  the  MRTD  is  obtained  by  using  the 
empirical  formula: 


MRTDcalc(r) 


c  ■  NETD  ■  FFe 
MTF  (r)  •  MTFe  (r) 


where  FFe(r)  and  MTFe(r)  are  the  eye  filter  function  and  the  eye  MTF;  c  is  a  constant  determined  by 
comparison  of  MRTDcaic(r)  with  a  subjectively  measured  MRTD. 


4.8  MATCHED  FILTER  TECHNIQUE 

This  technique  assumes  that  the  eye-brain  channel  acts  like  a  matched  filter  in  order  to  perceive  a  signal 
buried  in  noise.  The  signal  is  perceived  if  the  signal-noise  ratio  at  the  output  of  the  matched  filter  exceeds 
a  threshold.  These  assumptions  are  the  same  as  in  most  analytical  performance  models. 

The  matched  filter  approach  applied  to  the  perception  of  the  displayed  4-bar  pattern  images  assumes 
that  the  signal-noise  ratio  at  the  output  of  the  matched  filter  is  proportional  to  the  MRTD  or  MTDP  of  the 
sensor.  The  technique  consists  in  constructing  the  matched  filter  which  is  then  used  to  filter  the  displayed 
images  of  the  4-bar  pattern.34 

To  obtain  the  matched  filter,  images  of  the  4-bar  test  pattern  are  recorded  for  each  pattern  frequency 
at  high  temperature  differences.  If  necessary,  several  frames  are  averaged  to  further  minimize  the  noise. 
One  dimension  (ID)  and  two  dimension  (2D)  match  filters  can  be  constructed.  The  filter  is  then  applied  to 
a  series  of  low  contrast  images  that  are  obtained  over  a  range  of  small  temperature  differences,  where  the 
actual  range  of  temperature  differences  depends  on  the  sensor  under  test.  In  addition  to  the  low  contrast 


34 


bar  patterns,  images  of  an  open  position  on  the  filter  wheel  are  also  acquired  at  each  temperature.  These 
images  are  used  to  determine  the  noise. 

Algorithm  MRTD  and  MTDP  values  are  compared  to  subjective  values  obtained  by  human 
observers  in  Fig.  4.10.  MRTD  values  were  calculated  based  on  matched  filters  corresponding  to  the 
pattern  (corrected  reconstructed)  and  its  derivative  (corrected  difference).  The  results  of  the  corrected 
difference  are  in  good  agreement  with  the  subjective  values,  but  more  research  is  required  to  validate  this 
approach. 


SPATIAL  FREQUENCY  (cycle/mrad) 


Fig.  4. 10:  Subjective  (open  circles)  and  corrected  difference  (solid  circles)  and  reconstructed 
(crosses)  MRTD  values  are  compared.  The  agreement  between  the  difference  and 
subjective  values  is  quite  good. 


Chapter  5 
Discussion 
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A  variety  of  measures  and  measurement  techniques  have  been  developed  or  evaluated  to  characterise  the 
performance  of  advanced  thermal  imagers  with  respect  to  their 

•  geometrical  (spatial)  resolution 

•  thermal  resolution  (responsivity  and  noise) 

•  overall  device  performance  characterisation. 

In  addition  to  the  new  overall  device  performance  figures  of  merit  which  depend  on  the  subjective 
assessment  by  a  human  observer,  alternative  “objectives”  techniques  have  been  developed  and  discussed. 
Performance  measures  developed  for  1st  generation  thermal  imagers  which  can  also  be  used  without 
modifications  for  advanced  systems  were  not  addressed. 

5.1  GEOMETRICAL  RESOLUTION 

Measures  and  techniques  developed  or  investigated  to  characterise  the  geometrical  resolution  are 
summarised  in  Tabel  5.1.  The  following  techniques  appear  to  be  of  special  interest:  The  scanning  slit 
method  is  the  most  accurate  technique.  It  can  be  used  to  assess  the  system  in  any  direction  and  can  be 
applied  to  perform  measurements  in  the  digital  and  analog  video  signals  as  well  as  from  the  display.  The 
double  slit  technique  is  a  measure  of  the  minimum  distance  at  which  the  modulation  depth  reaches  a 
certain  value.  AMOP  is  an  overall  resolution  measure  that  is  directly  linked  to  the  standard  4-bar 
technique. 


Technique 

Advantages 

Disadvantages 

Best,  worst  ‘MTF’ 

fast;  simple  assessement  of 
undersampling 

no  information  on  MTF 

Scanning  Slit 

very  accurate 

not  affected  by  spatial  noise 

Tilted  Slit 

low  number  of  data 
correction  of  spatial  noise 

Discrete  MTF 

elegant 

digital  video  output  only 
correction  of  spatial  noise 

Double  Slit 

similar  to  Rayleigh  criterion 

no  information  on  MTF 
special  test  pattern 

Generalized  LSF 

special  test  pattern 
low  number  of  MTF  data 

Fractal  technique 

complicated  test  patterns 

AMOP 

overall  performance 
measure  incl.  undersampling 

setting  of  optimum  phase 

Table  5.1:  Measures  and  techniques  for  determining  the  geometrical  resolution  of  advanced  thermal 
imagers 
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5.2  THERMAL  RESOLUTION 

Proposed  and  investigated  measures  and  techniques  to  assess  thermal  resolution  are  summarised  in 
Table  5.2 

NETD  and  IETD  are  simple  measures  characterizing  temporal  and  random  fixed  pattern  noise  but, 
presently,  a  standardized  definition  of  these  terms  is  missing.  With  the  3-D  noise  and  the  principle 
components  techniques  all  possible  noise  terms  in  thermal  imagers  can  be  characterised.  The  principle 
components  technique  appears  to  be  the  most  promising  since  it  allows  to  identify  the  noise  sources  in  the 
imager.  The  characterization  of  bad  pixels  and  of  1/f  noise  is  an  important  aspect  in  assessing  thermal 
imager  noise  for  which  special  assessment  techniques  are  now  available. 


Technique 

Advantages 

Disadvantages 

NETD 

input  to  performance  models 

different  definitions  exist 

IETD 

input  to  performance  models 

not  standardized 

3  D  Noise 

full  noise  characterisation 

complex  mathematical 
analysis 

Schultz 

basis  for  bad  pixel  and  1/f 
noise  assessment 

Principle  components 

identifies  principal  noise 
contributions  in  the  imager 

complex  mathematical 
analysis 

Bad  pixel  and  1/f  noise 
characterisation 

quantitative  basis  for  detector 
specification 

Table  5.2:  Measures  and  techniques  for  determining  the  thermal  resolution  of  advanced  thermal 
imagers 


5.3  OVERALL  DEVICE  PERFORMANCE 

Subjective  overall  device  performance  figures  of  merit  are  summarised  in  Table  5.3. 


Technique 

Advantages 

Disadvantages 

Dynamic  MRTD 

uses  classical  resolution 
criterion 

not  related  to  other  performance 
measures 

MTDP 

evolution  of  the  MRTD 
concept;  can  be  predicted 
and  is  input  for  range 
performance  modelling 

TOD 

less  subjective  due  to  forced 
choice  method 

no  standard  4-bar  test  pattern; 

MRID 

resolution  defined  similar  to 
the  Rayleigh  criterion 

still  in  its  infancy;  relation  with 
range  performance  not  yet 
shown 

Table  5.3:  Figures  of  merit  and  techniques  for  determining  the  overall  device  performance  of 
advanced  thermal  imagers 
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Two  of  them  are  based  on  the  perception  of  the  standard  4-bar  test  pattern,  of  which  the  MTDP  is 
definitely  the  most  promising.  It  is  an  evolution  of  the  MRTD  concept  which  takes  into  account  the 
sampling  effects  of  the  imager.  The  double  slit  method  and  the  TOD  techniques  have  more  recently  been 
developed.  TOD  uses  triangles  as  test  patterns.  TOD  is  a  forced  choice  technique  where  the  observer 
responses  can  be  verified,  thus  obtaining  less  subjective  results.  The  MRID  is  still  in  its  infancy.  These 
techniques  have  benefits  which  should  further  be  investigated.  All  listed  figures  of  merit  probably  will 
remain  in  use  for  further  evaluation,  which  is  strongly  recommended. 

The  two  objective  assessment  techniques  that  are  summarised  in  Table  5.4  both  have  attractive 
features.  These  and  similar  techniques  are  still  under  development.  They  are  listed  here  in  order  to 
document  corresponding  TGI 2  activities  which  respond  to  the  continued  interest  of  the  IR  community  to 
have  fast  and  reliable  assessment  techniques  without  human  observer. 


Technique 

Advantages 

Disadvantages 

LSF  +  Noise 

extracted  from  resolution  and 
noise  measurements 

eye- brain  model  required; 

must  be  calibrated  with  subjective 

measurement 

Matched  filter 

suitable  for  all  types  of  imagers 
and  test  patterns 

eye-brain  model  required; 

must  be  calibrated  with  subjective 

measurement 

Table  5.4:  ‘Objective’  assessment  techniques 
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Chapter  6 

Conclusions  and  Recommendations 
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The  infrared  community  has  entered  an  era  where  staring  imagers  are  becoming  the  norm  and  not  the 
exception.  Staring  array  sensors  are  quickly  making  their  way  into  target  acquisition  and 
surveillance/reconnaissance  applications.  The  large  detector  characteristics  and  improved  optical  quality 
of  these  systems  make  for  undersampled  image  transfer  with  high  sensitivity.  The  classical  MTF  and 
MRT  resolution  and  visual  performance  parameters,  respectively,  do  not  exist  for  undersampled  imagers. 
In  addition,  the  higher  sensitivity  of  these  systems  requires  that  new  noise  measures  be  implemented. 

TG12  collectively  and  TG12  member  countries  have  developed  and  investigated  a  number  of 
measures  and  measurement  techniques  to  characterize  advanced  thermal  imagers.  These  measures  and 
techniques  allow  assessment  of  basic,  system-relevant  sensor  parameters,  such  as  spatial  resolution  and 
noise,  as  well  as  overall  sensor  performance. 

Numerous  measures  and  techniques  are  available  to  characterise  spatial  information  transfer  and 
noise  in  advanced  thermal  imagers.  Also,  various  figures  of  merit  and  measurement  techniques  are 
developed  to  assess  the  overall  device  performance.  Some  of  the  new  figures  of  merit,  such  as  MTDP,  are 
evolutions  of  the  classical  MRTD  figure  of  merit.  Other  techniques  for  overall  performance  assessment 
use  triangles  and  double-slits  as  test  patterns.  These  techniques  were  developed  to  reduce  possible 
measurement  inaccuracies  introduced  by  the  subjective  nature  of  assessment  techniques  which  use 
standard  4-bar  test  patterns.  They  have  benefits  which  should  further  be  investigated. 

Techniques  to  assess  overall  device  performance  without  human  observers  respond  to  the  continued 
interest  of  the  IR  community  to  have  fast  and  reliable  assessment  techniques.  Such  techniques  are  still  under 
development. 

It  is  recommended  that  those  STANAGs  which  address  performance  assessment  of  thermal  imagers  be 
reviewed.  This  covers  Part  III  Section  7  of  STANAG  4161  on  the  OTF  of  thermal  imaging  systems  as  well 
as  STANAGs  4347,  4349  and  4350  which  are  based  on  MRTD.  It  is  also  recommended  to  standardise 
definitions  and  measurement  techniques  for  assessing  the  noise  performance  of  advanced  imagers. 

Short  term  candidates  might  be  the  MTDP  approach  or  the  NVTherm  coupled  to  DMRT  approach.  It 
is  the  opinion  of  some  TG12  members  that  an  all-objective  measurement  of  resolution  and  sensitivity  and 
that  these  measurements  be  combined  to  replace  the  overall  visual  performance  encompassed  by  MRT. 
MRT  is  meaningless  in  an  undersampled  imager.  In  the  longer  term  the  forced  choice  TOD  method  might 
be  a  candidate,  which  implies  however  a  drastic  reconfiguration  of  the  targets  and  procedures  used.  Also  the 
double  slit  method,  after  careful  validation  with  field  experiments,  looks  promising  by  its  simplicity  and 
ease  of  modelling. 


Summary  of  Recommendations: 

1.  Review  and  update  STANAGs  4347,  4349,  and  4350  for  use  with  well-sampled  imaging  systems. 
There  have  been  numerous  improvements  in  MRT  modeling,  measurements,  and  range 
performance  over  the  past  10  years. 

2.  Establish  a  board  of  experts  and  an  AGGRESSIVE  program  for  developing/evaluating  techniques 
for  modeling,  measurements,  and  range  performance  predictions  of  undersampled  imagers  with  the 
goal  of  drafting  STANAG(s)  for  imager  performance.  Realize  that  this  is  a  significant  effort  that 
requires  nations  to  participate  in  laboratory  evaluations  of  techniques,  so  nations  must  be  willing  to 
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invest  time  and  effort  in  a  development/evaluation  program.  It  should  not  be  the  part-time  goal  of  a 
Technical  Group. 

3.  Note  that  for  some  immediate  future,  there  will  be  mis-applications  of  the  current  STANAG 
methods  to  undersampled  imagers.  Staling  and  scanning  infrared  sensors  are  unfairly  compared 
using  the  current  STANAG  methods.  It  is  recommended  that  the  MTDP  method  or  the 
NVTherm/DMRT  (dynamic  MRTD)  method  be  used  until  further  evaluation  can  be  performed. 

4.  It  is  also  recommended  to  standardise  definitions  and  measurement  techniques  for  assessing 
resolution  and  sensitivity  of  advanced  imagers. 

5.  The  development  of  “objective”  techniques  to  assess  overall  device  performance  without  human 
observer  respond  to  the  continued  interest  of  the  IR  community  and  should  further  be  encouraged. 
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